Computer systems may employ a multi-level hierarchy of memory. A relatively fast, expensive but limited-capacity memory may be provided at a highest level of the hierarchy and a relatively slower, lower cost (but higher-capacity) memory may be provided at the lowest level of the hierarchy. The hierarchy may include a small fast memory called a cache, either physically integrated within a processor or mounted physically close to the processor for speed. The computer system may employ separate instruction caches and data caches.
When executing an instruction that requires access to memory (e.g., read from memory or write to memory), a processor may access a cache in an attempt to satisfy the instruction. Of course, the cache may be implemented in a manner that allows the processor to access the cache in an efficient manner. That is, the cache may be implemented in a manner such that the processor is capable of accessing the cache (i.e., reading from the cache or writing to the cache) quickly so that the processor may execute instructions quickly. Caches have been configured in both on-chip and off-chip arrangements. On-chip caches have less latency, since they may be closer to the processor. However, since on-chip area is expensive, on-chip caches are typically smaller than off-chip caches. Off-chip caches may have longer latencies since they may be remotely located from the processor. However, such caches may be larger than on-chip caches.
Some systems have multiple caches, some small and some large. The smaller caches may be located on-chip, and the larger caches may be located off-chip. Typically, in multi-level cache designs, a first level of cache (i.e., an L0 cache) may be accessed first to determine whether a true cache hit for a memory access request is achieved. If a true cache hit is not achieved for the first level of cache, then a determination may be made for the second level of cache (i.e., an L1 cache), and so on, until the memory access request is satisfied by one of the levels of cache. If the requested address is not found in any of the cache levels, the processor may send a request to the system's main memory in an attempt to satisfy the request.